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SUMMARY 



PROGRAM DESCRIPTION 

The New York City (N.Y.C.) Board of Education's Division of 
Curriculum and Instruction was awarded a federal grant to conduct 
the Participation in Government Issues and Values Project for 
the 1986-87 school year and the fall, 1987 term. The State 
Education Department (S.E.D.) defined the main thrust of the 
Issues and Values course as teaching twelfth-grade students the 
proper skills and attitudes necessary for informed and active 
participation in government. Through the use of knowledge gained 
in tenth and eleventh grade history courses, the students were 
expected tc analyze the moral and ethical impact of laws and 
policies and subsequently participate in in-school and out-of- 
school government and advocacy activities as developed by 
individual teachers. During the fall, 1986 term 35 social 
studies or history teachers from 18 public high schools 
throughout N.Y.C. volunteered to be trained in law-related 
issues, values, concepts, and methods of instruction. A seven- 
unit curriculum model focusing on local, state, national, and 
global issues was also developed at this time. During the 
spring, 1987 term the trained teachers taught 55 sections of the 
Issues and Values Project course to approximately 1,200 students. 
Throughout the project period project administrators established 
over 20 high school and seven law-related education agency 
partnerships as well as attended six legal and educational 
professional conferences to implement the project's goals. 

PROGRAM GOALS 

The Issues and Values Project was designed to develop and 
implement a new law-related education curriculum for all N.Y.C. 
private and public high school twelfth-graders. This project was 
implemented as a pilot project to be modified and incorporated 
into a school's curriculum at a later date. Project 
administrators hoped to use the pilot project to: establish 26 
public and nonpublic high school and eight law-related education 
agency partnerships; train 52 public and nonpublic twelfth grade 
social studies teachers in a new law-related education approach; 
develop and disseminate a curriculum model with the new public 
policy issues approach to law-related education; and improve 
twelfth-grade students' knowledge and attitudes toward government 
participation and the law as measured by program-developed 
instruments administered at the beginning and end of the spring, 
1987 term. 



PROGRAM FINDINGS 

The project administrators were partially successful in 
meeting their program goals. They established 21 public and 
nonpublic high school and seven law-related educational agency 
partnerships. The nonpublic school teachers withdrew from the 



program shortly after it began because the Issues and Values 
curriculum did not fit into their schools' existing social 
studies. curriculum. The initial draft of the curriculum model 
was developed by the end of January, 1987 and finalized eleven 
months later at the end of December, 1987. The public policy 
focus was implemented in the curriculi^n with sufficient examples 
of how unit topics were developed for teachers to be able to make 
their own lessons if they desired. The majority of the Issues 
and Values Project course sections (67 percent) contained non- 
twelfth graders. According to the project coordinator, the 
partnership schools substituted the new course for their regular 
spring, 1987 government course sections which were predominantly 
for tenth and eleventh graders. As a result, most project 
classes included an introduction to United States government 
unit. This was done largely because of the pilot nature of the 
course. Project students* performance on the knowledge and 
attitude instruments indicated a slightly positive shift toward 
improvement from the pre-to posttest. 

CONCLUSIONS AND RECOMMENDATIONS 

The Issues and Values Project teachers were satisfied with 
the training they received and reacted favorably to the initial 
curriculum model which they taught in the spring, 1987 term.' 
However, the lack of active nonpublic school teachers* project 
participation prevented project administrators from fully 
implementing their new law-related education curriculum in all 
types of N.Y.C. twelfth-grade educational settings. In 
addition, because of the pilot nature of the course, the majority 
of the project students were tenth and eleventh graders. 
Finally, although project teachers reported that the course 
encouraged students to make decisions and participate, the lack 
of a control group of non-project students who completed the 
knowledge and attitude instruments made it impossible to assess 
whether changes were due to program partic3-pation or to other 
unrelated factors . 

On the basis of these evaluation findings, the following 
recommendations are aimed at enhancing program benefits and 
facilitating their assessment: 

o The curriculum model should include a comprehensive table 
of contents and stipulate course requirements indicating 
the minimum student skills, knowledge, behaviors, and 
out-of-class action projects necessary for passing each 
unit; 

o Project administrators and O.E.A. should work together to 
develop a strategy for assessing changes in students* 
attitudes or knowledge. This may involve both project 
students and a control group of non-project students. 
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!• INTRODUCTION 

PROGRAM BACKGROUND 

The New York City Board of Education's Division of 
Curriculum and Instruction was awarded a federal grant to conduct 
the Participation in Government Issues and Values Project 
(hereafter called the Issues and Values Project) for the 1986-87 
school year. It subsequently received a program extension through 
December 31, 1987. The overall purpose of the Issues and Values 
Project was to develop and institutionalize a new law-related 
education curriculum for twelf th~grade students. 

During the fall, 1986 term, 35 social studies/history 
teachers from 18 public high schools throughout N.Y.C. 
volunteered to be trained in law-related issues, values, 
concepts, and methods of instruction. Project administrators and 
other teacher volunteers developed curriculum materials for the 
new course at this time. During the spring term, the trained 
teachers taught the new social studies course to approximately 
1,200 students. Project administrators established over 20 public 
and nonpublic high school partnerships and seven law-related 
education agency partnerships to implement project goals. In 
addition, they attended numerous events specifically to 
distribute the project curriculum materials to legal and 
educational professionals throughout New York State (N.Y.S.). 
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PROGRAM OBJECTIVES 



The goals of the project which the Division of Curriculum 

and Instruction listed in the federal funding proposal were: 

o To establish partnerships with educational agencies made 
up of 20 public and six nonpublic schools, and eight 
public and private agencies promulgating law-related 
education; 

o To train 32 twelfth grade social studies teachers and 
eight special education high school teachers from 20 
public high schools and 12 twelfth-grade teachers from 
six nonpublic high schools in a new approach to law- 
related education with public policy issues, concepts, 
values and materials; 

o To develop a teaching/curriculum manual with the new 
public policy issues approach to law-related education; 

o To improve twelfth-grade public and nonpublic students' 
concepts and knowledge about the law; 

o To improve twelfth-grade public and nonpublic students' 
attitudes and values toward the law; and 

o To disseminate project objectives, activities, teaching 
methods, and a teaching/curriculum manual. 

After consultation with project staff, the Office of 

Educational Assessment (O.E.A.) redefined the above gocils into 

the following measurable objectives: 

o To identify the nature and number of partnerships 
established; 

o To determine whether 52 twelfth grade social studies 
teachers were trained in a new law-related education 
approach; 

o To document whether a teaching/curriculum manual was 
produced; 

o To determine whether students' knowledge of law-related 
concepts improved as measured by a program-developed 
questionnaire; 
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o To determine whether students' attitudes toward school 
and law-related values improved as measured by a program- 
. developed questionnaire; and 

o To document the number of dissemination events held. 
SCOPE OF THE EVALUATION 

O.E.A.'s evaluation of the Issues and Values Project was 
based on interviews with the project coordinator regarding the 
implementation of partnership and dissemination activities and 
the development of a curriculum manual. It also obtained 
teacher assessments of the training workshop:^ through surveys 
dealing with workshop presentations, curriculum materials, and 
teaching techniques. In addition, O.E.A. analyzed pre- and post- 
course student knowledge and attitude scores which it received 
from program staff. 
SCOPE OF THIS REPORT 

The report of the 1986-87 Issues and Values Project 
consists of four chapters. Program background and scope of the 
evaluation are contained in Chapter I. A description of program 
implementation is provided in Chapter II. An analysis of student 
outcomes on the -participation in government examination and 
attitude survey are included in Chapter III, while conclusions 
and recommendations are contained in Chapter IV. 
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II • PROGRAM IMPLEMENTATION 

PROGRAM OVERVIEW 

The State Education DepartiTient (S.E.D,) defined the main 
thrust of the Issues and Values course as teaching twelf th~grade 
students the proper skills and attitudes necessary for informed 
and active participation in government. Through the use of 
knowledge gained in world and United States history courses, the 
students were expected to analyze the moral and ethical impact of 
local, state, nationals and global laws and policies and 
subsequently participate in activities that affect government 
decision-making. The S.E.D. identified these st dent activities 
as involvement in advocacy, school government, election 
campaigns, and local government, as well as classroom discussions 
of governmental policy issues. Teachers had the flexibility to 
develop the program in a way that was best suited to their 
particular students within the context of the program's 
objectives . 

In several important areas, administrators had to narrow the 
project's scope. Initially, they established partnerships with 
three nonpublic N.Y.C. high school systems and 22 public N.Y.C* 
high schools.* Staff from these institutions agreed to identify 
two teachers with two classes each to participate in the 
project's training and instructional components and to receive 
curriculum inaterials. Project administrators had anticipated that 



*See Appendix A for Ihe listing of high school partnership 
members . 
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52 social studies/history teachers, including eight special 
education, instructors, would volunteer. However, only 33 public 
and two nonpublic school teachers responded; of these, only three 
were special education teachers. The nonpublic school teachers 
withdrew from the program shortly after it began because the 
Issues and Values curriculum did not fit into their schools' 
existing social studies curriculum. 

Although the project was designed specifically for 
twelf th~grade students, during this pilot year only 33 percent of 
project classes included twelfth graders. Most of the 
participants were in the tenth and eleventh grades. According to 
the project coordinator, the partnership schools substituted the 
new participation in government course for their regular spring, 
1987 government course sections during this pilot year. In that 
most of these course sections were for tenth and eleventh 
graders, the majority of the project students brought much more 
limited social studies knowledge to the Issues and Values class 
than did twelfth graders. This resulted in an introduction to 
U.S. government unit being included in the new courses 's 
implementation. The project is subsequently expected to be part 
of the twelfth-grade curriculum. 

Project administrators had suggested several options that 
partnership schools could undertake so students could gain 
practical experience related to the curriculum. Teachers 
frequently assigned such classroom and in~school activities as 
student government participation, letters to senators, conflict- 
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resolution, and public policy decision-making exercises. A few 
teachers had students visit police precincts, senior citizen 
centers, cr community-based organizations; most of these 
assignments were related to term paper projects. Very limited 
out-of-school activities were done because teachers reported that 
their students were hesitant to travel long distances due to 
their concerns about safety and a lack of travel experience 
outside of their immediate neighborhood. 

TRAINING 

There were a total of 15 two-hour workshop sessions held 
during the fall, 1986 term. The workshop series was designed to 
expose the teachers to presentation, curriculum materials and 
teaching techniques that would prepare them to conduct an Issues 
and Values course. Workshop topics included: 

o Regents action plan; 

o Values education and citizenship; 

o Methods for making public issues understandable to 
students ; 

o Ways of resolving conflicting issues and values 
within the constitutional system; and 

• o Drugs and individual rights. 

Project administrators established partnerships with seven law- 
related education agencies who provided most of the workshop 
leaders. Four agencies were private: Syracuse University's 
Maxwell School o,f Politics; the Kettering Foundation, Public 
Agenda Foundation and Project Legal; and three were public: 
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Jericho School District, Liverpool High School, and N.Y.C.'s 
Victim Services Agency, Teen Crime Prevention Project. 

The teachers reported their workshop assessment at the end 
of each session on an evaluation form. The workshops were 
evaluated in three general areas: presentation, curriculum 
materials, and teaching techniques. Teachers responded to each 
item on a scale ranging from "very poor" (one) to "very good" 
(five) . After initial negative feedback program planners became 
concerned about teachers' perceptions of the quality of training 
sessions and decided to use workshop evaluations as formative 
feedback to improve the quality of subsequent workshops. 

As such, program staff expected that the teachers' 
perceptions of the quality of the workshops would be more 
favorable as the series progressed. In order to evaluate 
whether their perceptions improved O.E.A. compared a workshop 
session near the beginning of the series and near its end. This 
comparison indicated that teachers' perceptions remained 
relatively stable and positive throughout the workshop series. 
CURRICULUM DEVELOPMENT 
Initial Draft 

The project coordinator and seven N.Y.C. public high school 
social studies teacher volunteers developed the Issues and Value: 
Project curriculum model during the fall, 1986 term. The 
project coordinator selected teacher members of the curriculum 
development committee on the basis of recommendations by their 
chairpersons and their previous curriculum development 



experience. The committee used the S.E.D.'s course objectives, 
as well as the training partnership agencies* workshop materials, 
to design a total of seven units: 

1. Responsibilities of citizens; 

2. Obscenity; 

3. Crime as a national issue; 

4. Drug abuse; 

5. The homeless; 

6. Human rights; and 
1. Immigration. 

The committee completed the first draft of the curriculum manual ^ 
by the beginning of the spring, 1987 term. Unit topics reflect 
local, state, national, or global issues. For example the Unit 
One topic: "Are citizenship rights more important than 
responsibilities?"; is a national issue. Whereas, the Unit Two 
topic: "Should obscene materials be permitted to be sold in the 
local community?" reflects a local focus. In addition, "Should 
American foreign policy be committed to human rights?" and "Can 
America formulate- a workable immigration policy for the 21st 
century?" are global issues presented in Units Six and Seven, 
respectively. 

The size of the units range from four to 16 topics. The 
topics in each unit are self-contained and cover a particular 
issue. Every topic includes a variety of elements: social studies 
facts, civic values, concepts, analytical skills, student 
performance objectives, and action projects. The elements in each 
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unit are clearly labeled, so teachers can use them to develop 
their own. policy or issue analysis lessons. Topics are presented 
in the form of case studies, pro and con arguments, and 
provocative questions • The presentation of such concepts as 
change, choice, power, and authority helps students explore 
liberty, health, justice, equality, and other values. It also 
helps them build their decision-making, analytic, and 
interpretive skills. Because the unit topics are self-contained, 
teachers can select whichever ones their students are interested 
in or they feel competent to teach. The curriculum contains no 
list of topics which every teacher must teach nor a table of 
contents. Moreover the majority of the action projects are 
school-based rather than focused on student participation in 
government and advocacy institutions. 

Curriculum Revision 

Throughout the project term the curriculum development 
committee received feedback on its curriculum draft. Project 
administrators attended six legal and educational professional 
conferences held within N.Y.S. to provide information on how to 
develop a one-term twelfth-grade issues and values government 
participation course and receive reactions to their curriculum 
model. The conferences were sponsored by the following 
organizations : 

o New York State Council of Social Studies; 

o National Council for the Social Studies; 
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o Social Studies Association of New York City; 

•o Association of Teachers of Social Studies; 

o Social Studies Supervisors of the United Federation 
of Teachers; 

o New York State Bar Association; and 
o New, York State Education Department. 

Project teachers made several presentations to their 
schools' parent-teachers association meetings for parents 
reactions to the curriculum. In addition, project teachers also 
responded to an open-ended quest? onnaire evaluating the 
curriculum after tney had taught the course in the spring, 1987 
term. Seventeen of the 33 teachers assessed curriculum strengths 
and made recommendations for curriculum changes. Table 1 presents 
their responses. The majority of respondents believed that the 
curriculum was successful in motivating student decision making 
and government participation. A quarter characterized the 
entire curriculum as strong and 18 percent were pleased at the 
opportunity to use innovative types of presentations. On the 
whole, the teachers were impressed with the curriculum. 

Teachers recommended several curriculum changes. Twenty 
percent of respondents favored including a review of government 
course as an introductory unit; the relatively high percentage 
making this recommendation was due to the large number of tenth 
and eleventh graders enrolled in the project classes. Twenty 
percent also wanted classes to be more participatory and 15 
percent recommended developing an out-of-class participation 
program for students. These two recommendations reflect the 
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Table 1 



Summary of Teachers* Post-Course Reactions 
To Issues and Values Curriculum 



Reaction Area N 



Curriculum Strengths^ 

Encourages students to make decisions 

and participate 9 53 

The entire curriculum model 4 24 

Provides an opportunity to use innovative types 

of presentations and materials 3 18 

It is only good for second term seniors 1 6 

The unit on citizen rights and responsibilities 1 6 

Recommendations for Curriculum Changes^ 

Include a review of government course as an 

introductory unit 4 20 

Presentations should be participatory, not 

lectures 4 20 

Develop an out of class participation program 

for the students 3 15 

Provide more preparation time before teaching 

the course 2 10 

Develop a guide of participatory teaching 

approaches 2 10 

Develop an on-going process to include new 

issues in the course 2 10 

Provide issues that students feel comfortable 
challenging 1 5 

Students should be twelfth graders 1 5 

The course should only have three units 1 5 



* The total number of responses in this area was 11* 

The total number of responses in this area was 20, 

<^ The majority of the teachers (53 percent) identified 
curriculum strengths that reflect the project's student 
participation objectives. However, their top three 
recommendations indicate their students lacked government 
knowledge and the curriculum had limited experiential 
learning options. 
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limited number of experiential learning options in the curriculum 
model. Finally, developing a guide of participatory teaching 
approaches for the course, allowing more preparation time before 
teaching the course, and creating a mechanism for including new 
issues in the curriculum were each recommended by 10 percent of 
'che respondents. 

As a result of the feedback gained from the dissemination 
meetings and project teachers' reactions the* curriculum 
development committee completed its curriculum revisions at the 
end of December, 1987. 



III. STUDENT OUTCOMES 
PROJECT STUDENT POPULATION 

Approximately 1,200 students were enrolled in 55 sections of 
the Issues and Values course during the spring, 1987 term. Six 
of these sections were special education classes and 57 contained 
only tenth and eleventh graders • 

PARTICIPATION IN GOVERNMENT EXAMINATION 

During the fall, 1986 term the Issues and Values Project 
administrators, in conjunction with O.E.A., developed a ten-item 
multiple-choice student knowledge examination of government 
participation rights and responsibilities. Questions dealt with 
such topics as the values that can affect political decisions, 
the meaning of due process, and the usefulness of public opinion 
polls. Students took a pretest in February, 1987 and a posttest 
in May, 1987. The project teachers submitted a total of 1,179 
pretests and 842 posttests from 15 and 16 schools, 
respectively.* A total of 410 students, (35 percent), took both 
the pre- and the posttests; only their data are used in this 
analysis of the change in student knowledge. The absence of 
complete and accurate student identification information limited 
the number of students whose data could be analyzed. Examination 
forms were returned so late in the spring, 1987 term it was 



* See Appendix B for a listing of the schools included in 
each sample. 
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impossible to obtain missing data and correct erroneous forms 
that were submitted. 

The students' knowledge of their govp nment participation 
rights and responsibilities generally remained stable from the 
pretest to the posttest. They correctly answered an average of 
six questions right on the posttest as compared with an average 
of five on the pretest indicating the educational meaning- 
fulness of the mean gain for this comparison was found to be 
0.2** Thus it can be concluded that the Issues and Values course 
did not greatly improve the students' knowledge of government 
participation rights and responsibilities as measured by this 
exam.* * 

ATTITUDE SURVEY 

O.E.A. also assisted the Issues and Values Project 
administrators in developing a student survey consisting of ten 
statements regarding beliefs and attitudes toward the impact of 
participating in government decision-making activities and 
exercising citizenship rights and responsibilities. Students 



*The effect size, developed by Jacob Cohen, si the ratio of 
the mean gain to the standard deviation of the gain. This 
ratio providews an index of improvement in standard 
deviation units irrespective of the size of the sample. 
According to Cohen, 0.2 is a small E.S., 0.5 is a moderate 
E.S., and 0.8 is considered to be a large E.S. Only effect 
sizes of 0.8 and above are considered to be educationally 
meaningful . 

** A more detailed analysis of the participation in 
government examination can be found in Appendix C. 
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responded on a five-point scale running from "strongly agree" 
(one) to "strongly disagree" (five) . 

Program teachers administered the same survey at the 
beginning (pretest) and at the end (posttest) of the spring , 1987 
term. Student names appeared on the completed surveys, A total 
of 1/171 pretests and 851 posttests were submitted. Only 409 
students (35 percent) responded to both the pretest and the 
posttest and therefore could be included in this analysis of 
student attitude changes. 

In general students expressed positive attitudes. The 
percentage of positive attitudes on the posttest was 83 as 
compared to 77 percent on the pretest.* The students' average 
rating on the posttest was "agree" (two) which represented a more 
positive attitude than the average pretest rating (three) . The 
S.S. for this gain was 0.4 which indicates that the Issues and 
Values course did not greatly improve the students' attitudes.** 



* O.E.A. computed reliability coefficients for the 
pretest and the posttest separately for the students V3,th 
complete data. Using a measure of internal consistency, 
alpha, and the Spearman Brown reliability estimate for 
the survey if it were lengthened to 100 items, the 
reliability coefficients were determined to be high, 
for both the posttest and the pretest (r = .90 and r = .89, 
respectively) . This indicates that, in general, the survey 
items were measuring the same thing. 

** A more detailed analysis of the attitude survey appears 
in Appendix C. 



IV. CONCLUSIONS AND RSCOMMENDATIONS 
The Issues and Values Project met its objectives of 
establishing public school and law-related agency partnerships. 
These relationships were also instrumental in helping project 
administrators achieve their teacher training objective. 
However r the lack of active nonpublic school teachers* 
participation prevented project administrators from fully 
implementing their new law-related education curriculum in all 
types of N.Y.C. educational settings. 

Project administrators were successful in developing and 
disseminating their curriculum model. The curriculum draft was 
well organized, but lacked a table of contents. Its structure 
encouraged teachers to develop their own lessons based on their 
own and students* interests. However, the absence of minimum 
criteria for course content meant that teachers at different 
schools could be teaching very different courses; and therefore 
students would not all cover the ^ame basic material. In 
addition, the lack of experiential learning options within the 
curriculum resulted in several recommendations for curriculum 
changes. 

The project student population was vastly different from 
what was planned. The project was designed for twelfth graders. 
Because the project was implemented in substitution for regular 
government course sections, the ,:ajority of the student 
participants were in the tenth and eleventh grades. The 
participation of these students, particularly of the tenth 
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graders, limited the effectiveness of the project. High school 
sophomores generally do not have the prerequisite knowledge of 
government structure to adequately evaluate the various public 
policy issues and values contained in the project curriculum. 
According to project staff, the change in the target population 
is largely due to the pilot nature of the course. The course is 
intended for twelfth graders and will be offered as a twelfth- 
grade course in subsequent years. 

Teacher assessments of both the training workshops and the 
curriculum were positive. However, because less than a third of 
the project students were twelfi:h graders the project teachers . 
were unable to fully implement what they had learned in 
training. Moreover the lack of student government participation 
internships resulted in a far less pragmatic approach to law- 
related education than the S.E.D. defined. 

The findings from the tests used in this project regarding 
student knowledge and attitudes were inconclusive because the 
testing procedures used to determine student outcomes were 
inappropriate. The lack of a control group of non-project 
students who completed the knowledge examination and the survey 
made it impossible to assess whether changes were due to program 
participation or to other unrelated factors. In addition, 
examination and survey forms asked for student names but not for 
grade level, nor for students* identification numbers. The 
inclusion of names on the attitude survey may have influenced how 
students responded. Whereas, the omission of the students' grade 

17 



level and identification numbers made it impossible to do a 



separate analysis of twelfth graders' outcomes. Moreover the lack 

of sufficient and accurate student identification data was the 

main reason only 35 percent of the examinations and surveys could 

be fully analyzed. 

On the basis of these evaluation findings, the following 

recommendations are aimed at enhancing program benefits and 

facilitating their assessment: 

o The curriculum model should include a comprehensive table 
of contents and stipulate course requirements indicating 
the minimum student skills, knowledge, behaviors, and 
out-of-class action projects necessary for passing each 
unit; 

o Project administrators and O.E.A. should work together to 
develop a strategy for assessing changes in students^ 
attitudes or knowledge. This may involve both project 
students and a control group of non-project students; and 

o Project administrators and O.E.A. should develop a more 
comprehensive and timely schedule of evaluation 
activities to ensure proper data collection, and 
analysis . 
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APPENDIX A: PARTNERSHIP SCHOOLS 
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PARTNERSHIP SCHOOLS 



Non Public High School Systems 

Agudath Israel America 
Archdiocese of New York 

Office of Catholic Education of the Diocese of Brooklyn 

Public High Schools 

Benjamin Cardozo High School 
Boys and Girls High School 
Franklin K. Lane High School 
Fort Hamilton High School 
High School for the Humanities 
James Madison High School 
John Jay High School 
John F. Kennedy High School 
Martin Van Buren High School 
Midwood High School 
New Utrecht High School 
Seward Park High School 
South Shore High School 
Theodore Roosevelt High School 
Thomas Jefferson High School 
Tottenville High School 
Walton High School 
Washington Irving High School 
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SCHOOLS INCLUDED IN EACH SAMPLE 



School Included Included Included 

Name in in in 

Pretest Posttest Matched Sample 



Benjamin Cardozo High School 
Boys and Girls High School 
Fort Hamilton High School 
Franklin K. Lane High School 
High School for the Humanities 
James Madison High School 
John F. Kennedy High School 
John Jay High School 
Martin Van Buren High School 
Midwood High School 
New Utrecht High School 
Seward Park High School 
South Shore High School 
Theodore Roosevelt High School 
Thomas Jefferson High School 
Tottenville High School 
Walton High School 
Washington Irving High School 



XX X 
X 

XX X 

XX X 

XX X 

XX X 

XX X 

XX X 

XX X 

XX X 
X 

XX X 

XX X 

X 

X 

XX X 

X 

XX X 
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APPENDIX C: ITEM ANALYSIS OF PARTICIPATION IN 
GOVERNMENT EXAMINATION AND ATTITUDE SURVEY 
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ITEM ANALYSIS OF PARTICIPATION IN GOVERNMENT 
EXAMINATION AND ATTITUDE SURVEY 



As part of its 1986-87 Issues and Values Project evaluation 
O.E.A. conducted an item analysis on the government participation 
examination and examined the reliability of the attitude survey. 
These analyses are summarized in this appendix, 

PARTICIPATION IN GOVERNMENT EXAMINATION 

• O.E.A. conducted an item analysis of the participation in 

government examination for the 409 students with complete pretest 
and posttest data. The same ten item test was administered for 
both the pretest and the posttest. All but two of the test items 
were four-choice multiple choice questions; the others were six- 
choice questions. It would be expected that students would 
answer the four-choice items correctly approximately 25 percent 
of the time, and the six-choice items correctly approximately 17 
percent of the time, by chance alone. 

Examination performance for the pretest is illustrated in 
Appendix Table 1. Students demonstrated some knowledge of the 
material prior to any classroom experience, with better than 
chance performance on nine pretest items. Item four had the 
highest percentage of correct responses (71), while item two had 
the lowest (19) . On average, the students answered 53 percent of 
the items correctly. Project students' posttest performance is 
represented in Appendix Table 2. Both item numbers two and four 
continued to be the lowest and highest correctly answered 
questions, respectively. Moreover, from Appendix Table 3 it can 
be seen that the average net percentage of change on all ten 
items was five percent. This overall lack of change in students' 
test performance is consistant with the lack of educational 
meaningf ulness found when the effect size was computed. 

ATTITUDE SURVEY 

The Issues and Values Project staff and O.E.A. developed a 
ten item student survey regarding beliefs and attitudes toward 
the impact of participating in government decision-making 
activities and exercising citizenship rights and 
responsibilities. Issues and Values Project staff administered 
the same test as- both pretest and posttest. Four-hundred and 
nine students had both pretest and posttest survey data and could 
therefore be included in the O.E.A. analysis. 

The pretest and posttest reliabilities were examined to 
determine whether the survey items represent different aspects of 
the same area or constitute more than one area. The designation 
for this type of reliability is coefficient alpha. Surveys with 
coefficient alphas of .60 or greater indicate that their items 
are from one area. Since reliability is more accurately 
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APPENDIX TABLE 1 



Summary of Students' Responses to Each of 
the Participation in Government Pretest Exam Items 



Item^ Number^ Number Percent 
Correct Incorrect Correct 



1 


179 


231 


44 


2 


79 


331 


19 


3 


260 


150 


63 


4 


292 


118 


71 


5 


201 


209 


49 


6 


236 


174 


58 


7 


199 


211 


49 


8 


250 


160 


61 


9 


214 


196 


52 


10 


269 


141 


66 



^ Items number five and six were six-choice multiple choice questions, 
all the rest of the exam items were four-choice multiple choice 
questions. 

^ There were 409 students' data included in this analysis. 

o As a group, the students correctly responded to 90 
percent of the items at a greater than chance level. 
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APPENDIX TABLE 2 

Summary of Students' Responses to Each of 
the Participation in Government Posttest Exam Items 



Item^ Number^ Number Percent 
Correct Incorrect Correct 



1 

J, 




1 ftfi 


55 


2 


94 


316 


23 


3 


257 


153 


63 


4 


313 


97 


76 


5 


229 


181 


56 


6 


253 


157 


62 


7 


236 


174 


58 


8 


251 


159 


61 


9 


235 


175 


57 


10 


270 


140 


66 



^ Items number five and six were six-choice multiple choice 
questions, all the rest of the exam items were four-choice 
multiple choice questions. 

^ There were 409 students' data included in this analysis. 

o As a group, the students' test performance remained 

basically the same from the pretest to the posttest: once 
again 90 percent of the items were answered correctly at 
a greater than chance level and items number two and four 
were the lowest and highest ranked items, respectively. 
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APPENDIX TABLE 3 



Net Change From Participation in Government Pretest 
to Ppsttest of Percentage of Students 
Answering Items Correctly 



Item % Change 



1 11 

2 4 

3 0 

4 5 

5 7 

6 4 

7 9 

8 0 

9 5 
10 0 



There was a positive change in 70 percent of the items, 
the other 30 percent remained the same; however the 
total change was not educationally meaningful. 
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for this type of reliability is coefficient alpha. Surveys with 
coefficient alphas of .60 or greater indicate that their items 
are from, one area. Since reliability is more accurately 
determined when using a long, as opposed to short survey, each 
test's reliability was estimated using the Spearman Brown 
formula. Thus the reported pretest and posttest reliabilities, 
r = .89 and r = .90, respectively, were computed as if the 
pretest and the posttest each contained 100 items • From these 
coefficient alphas it can be seen that the attitude survey items 
did in fact represent different aspects of the same area. 
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